feat(events): add block_gossip chain event - #535
Conversation
51184e6 to
5be589f
Compare
Add the block_gossip topic to the /lean/v0/events stream, the analog of the beacon block_gossip event: a block seen on the network before import (import may pend it while its parent chain is fetched). Emitted from the blockchain actor's NewBlock handler rather than a second P2P-side publisher, so the actor stays the sole event publisher and the write flow stays one-directional. Fires for both gossip'd and by-root-fetched blocks, since both re-enter through NewBlock. Ungated like block (not recency-gated), so subscribers can watch sync progress. Low-rate (one per block), so the payload is built unconditionally behind emit's no-subscriber guard.
e17223c to
a492c00
Compare
🤖 Kimi Code ReviewThis is a clean, straightforward addition of a No critical issues found. A few minor observations:
Code quality:
LGTM. Automated review by Kimi (Moonshot AI) · kimi-k2.5 · custom prompt |
🤖 Codex Code Review
I did not find additional consensus, security, or memory-safety issues in this diff itself. Verification note: I could not run the targeted Rust tests in this environment because Cargo/Rustup needs to write under read-only home directories ( Automated review by OpenAI Codex · gpt-5.4 · custom prompt |
Greptile SummaryAdds the beacon-style
Confidence Score: 4/5The PR appears safe to merge, with a non-blocking opportunity to avoid redundant block-root hashing when no event subscribers exist. The new event is wired through topic parsing, filtering, serialization, emission, tests, and documentation; the only accepted concern is avoidable payload computation before the event bus can take its no-subscriber fast path. crates/blockchain/src/lib.rs
|
| Filename | Overview |
|---|---|
| crates/blockchain/src/events.rs | Adds complete topic parsing, naming, event mapping, and unit coverage for BlockGossip. |
| crates/blockchain/src/lib.rs | Emits BlockGossip before import, but computes the block root even when the event bus has no subscribers. |
| docs/rpc.md | Accurately documents the new SSE topic, payload, and emission timing. |
Sequence Diagram
sequenceDiagram
participant P2P
participant BC as Blockchain actor
participant Bus as Event bus
participant Import as Block import
participant SSE as SSE subscriber
P2P->>BC: NewBlock
BC->>BC: Compute block root
BC->>Bus: emit(block_gossip)
Bus-->>SSE: event: block_gossip
BC->>Import: on_block
Import->>Import: Compute block root and import or pend
Prompt To Fix All With AI
Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 1
crates/blockchain/src/lib.rs:1327
**Avoid redundant block-root hashing**
The block root is computed before `EventBus::emit` can apply its no-subscriber guard, so handling `NewBlock` without SSE subscribers computes and discards a full SSZ root before normal block processing computes the same root again. Exposing a lazy emission path or reusing the root would avoid this actor-thread CPU cost.
Reviews (1): Last reviewed commit: "Merge branch 'main' into feat/events-blo..." | Re-trigger Greptile
Continues the chain-event pub-sub series (#516 / #517 / #518, all merged). Adds the two high-rate topics. ## Events added | Topic | Payload | Emitted when | |-------|---------|--------------| | `attestation` | `{ validator_id, data: AttestationData }` | A single validator vote passes gossip validation (signature omitted) | | `aggregate` | `{ participants: [u64], data: AttestationData }` | A committee-signature aggregate is produced locally or accepted from gossip (proof omitted) | Both fire only for messages the store **accepted** (data + signature validation), so subscribers see the same votes fork choice does. A node never receives its own aggregate back over gossip, so `aggregate` fires at two sites: the locally produced `AggregateProduced` path and the gossip-received path. ## Design points - The ~3 KB XMSS signature and the SNARK proof bytes are deliberately **omitted** — too heavy for a high-rate stream. - The shared broadcast channel capacity is bumped **256 → 8192**. All topics share one ring buffer, so a subscriber's tolerable stall is `capacity / total_event_rate` (dominated by the attestation rate), not per-topic. A per-topic split behind the `EventBus` facade remains the escape hatch if real devnet rates show the shared window biting — see the note in `docs/rpc.md`. - Emission uses the plain, no-subscriber-guarded `emit`. Consequence: the per-vote `AttestationData` is cloned even when nobody is subscribed (the guard only skips the send). If that actor-hot-path cost proves material, a payload-deferral helper can be added in a follow-up. ## Testing `cargo fmt`, `cargo clippy -p ethlambda-blockchain -p ethlambda-rpc` (clean), blockchain + rpc lib tests pass, including `events_streams_attestation_with_nested_data` (verifies the untagged `data:` nesting over the wire). --- **Independent PR, based off `main`.** One of three sibling PRs continuing the series — alongside #533 (`chain_reorg`/`safe_target`) and #535 (`block_gossip`), each independently based off `main`. They touch overlapping regions of `events.rs` / `docs/rpc.md`, so whichever two merge later will each need a small conflict rebase.
## Summary Adds `tooling/event-monitor/`, a standalone live arrival-time dashboard for lean-consensus (ethlambda) nodes. It dials the `GET /lean/v0/events` SSE stream of several nodes, timestamps each event on a single collector clock, and serves a browser dashboard visualizing: - a **rolling beeswarm** of each event's arrival offset *within the slot*, one lane per node, and - a **propagation-delta beeswarm**: for a given block / aggregate, how long after the *first* node each other node saw it. The rolling window is adjustable live from the header, and a fresh page load backfills recent history from the collector so it is never blank. ## Design - **Standalone Cargo workspace** under `tooling/event-monitor/` (empty `[workspace]` table); it is not a member of the parent `ethlambda` workspace and does not affect the main build. - **Zero dependency on any ethlambda crate** — it only speaks the documented SSE/HTTP wire shape. `CONTRACT.md` is the frozen interface between the Rust/axum backend (`src/`) and the vanilla-JS frontend (`web/`, no build step). - A single collector clock makes propagation deltas skew-free across nodes. - `?demo=1` runs the frontend fully offline with synthetic data. This PR contains the tooling plus the CI wiring needed to actually check it. The RPC / event-bus changes that make a node emit these events shipped separately in the chain-events series (below). ## Required ethlambda API functionality All dependencies are **satisfied on `main`** — the default dashboard view works against a current node with no extra PRs. | Capability | Endpoint / topic(s) | Status | |---|---|---| | SSE events stream | `GET /lean/v0/events` | ✅ #517 | | Topic filtering | `?topics=<csv>` | ✅ #518 | | Slot-geometry bootstrap | `GET /lean/v0/genesis`, `GET /lean/v0/config/spec` | ✅ | | Base topics | `block`, `head`, `justified_checkpoint`, `finalized_checkpoint` | ✅ #516 | | Attestation / aggregate topics | `attestation`, `aggregate` | ✅ #534 | | Block-gossip topic | `block_gossip` | ✅ #535 | The collector accepts `block`, `block_gossip`, `head`, `justified_checkpoint`, `finalized_checkpoint`, `attestation` and `aggregate`; any other topic is logged and skipped. `safe_target` / `chain_reorg` (#533, closed) were removed in 4eeafff — both panels' topic filters discarded them, and `chain_reorg` is a point-in-time event rather than a per-node arrival race, so it does not belong on a beeswarm. Both panels default to `block` / `attestation` / `aggregate`. ## Review fixes Three defects found in review, each with a regression test: - **Reconnect backoff never reset after a healthy session.** The reset was keyed on a clean end-of-stream, but a node restart surfaces as a stream *error*, so every restart ratcheted the delay one step permanently; after ~7 of them a healthy node reported `down` with 10s reconnects. Now keyed on session duration (survived ≥1 heartbeat). This also closes the inverse case: a peer that accepts the request and instantly closes the stream used to be retried at `INITIAL_BACKOFF` forever. - **One bogus slot blanked the dashboard.** Both the history ring and the rolling window key retention off the highest slot seen, and that watermark only moves up, so a single event from a node on a different genesis aged out every real event until a restart. Events more than `MAX_FUTURE_SLOTS` ahead of the collector's own slot are now dropped, warning once per node per connection. Only the future side is bounded; old slots are legitimate (`finalized_checkpoint` trails head) and cannot move the watermark. - **Stale status clobbered fresh status.** The frontend applies `status` immediately but buffers `chain` during backfill, so the `/api/history` snapshot could overwrite a newer live status. Live status now wins. Also from that review: - **Slot geometry is re-resolved every 60s** and republished when it changes, dropping the retained history and slot watermark, so a regenerated genesis no longer silently corrupts every subsequent `offset_ms`. Collectors re-read per frame and `/api/meta` per request; an already-open tab needs a reload to pick up new geometry. - **The arrival axis now spans two slots at two scales**: the first at full resolution (90.9% of the width at 5 intervals), the next compressed into a tinted band half a first-slot interval wide, saturating beyond that. A block spilling past its slot boundary is the failure mode worth seeing, and it used to be indistinguishable from one landing exactly on the boundary. - **`offset_ms` includes the collector↔node round trip.** One clock is what makes propagation deltas skew-free, but nodes reached over different links carry a systematic offset that reads as lag. Now documented in `README.md` and `CONTRACT.md §2`. - History events are `Arc`-shared, so `publish_chain` no longer deep-copies per event and `/api/history` no longer clones up to `HISTORY_MAX_EVENTS` events while holding the mutex. - `topics = []` / `nodes = []` are rejected at load rather than becoming an opaque retry loop against a 400. Known and deliberate, left as follow-ups: the collector cannot see upstream `: error - dropped N messages` comments (`eventsource-stream` swallows them), so a lagging subscription silently under-samples; both canvases still repaint unconditionally at 60fps; and `bootstrap` has no retry, so the monitor exits if started before its nodes are listening. ## CI `tooling/event-monitor` declares its own `[workspace]`, so `cargo fmt --all`, `cargo check --workspace`, `cargo clippy --workspace`, `make lint` and `make test` all stop at the root workspace members and never reached it — it landed with nothing verifying it. The `Lint` job now also runs `cargo fmt --check`, `cargo clippy --all-targets -- -D warnings` and `cargo test` under `working-directory: tooling/event-monitor`, and `rust-cache` lists it as a second workspace so its target dir is cached and its `Cargo.lock` feeds the cache key. ## Testing - `cargo test` in `tooling/event-monitor/`: **41 passed** (39 lib + 2 integration), 0 failed. - `cargo fmt --check` and `cargo clippy --all-targets -- -D warnings`: clean. - `?demo=1` offline synthetic mode renders both panels with no live nodes. - Verified end-to-end against a live local devnet (blocks land ~0.1–0.2 s into the slot, attestations ~0.8 s = interval 1). - Two-slot axis geometry checked numerically: the overflow band is exactly 0.5 × one first-slot interval, the mapping is monotonic, and the endpoints land on the margins. - Geometry refresh checked against a stub node: rewriting its `genesis_time` mid-run is picked up within one refresh interval (`WARN slot geometry changed`), `/api/meta` then serves the new epoch, and the history ring holds the restarted chain's low slots with in-slot offsets — which only works because the watermark is reset, otherwise they would prune as older than the previous high-water mark. ## Run ```bash cd tooling/event-monitor cp config.example.toml config.toml $EDITOR config.toml # list your nodes' RPC URLs + topics cargo run --release -- --config config.toml # open the `listen` address (default http://127.0.0.1:8080) ```
Continues the chain-event pub-sub series (#516 / #517 / #518, all merged). Adds the beacon
block_gossipanalog.Event added
block_gossip{ slot, block }Notes
NewBlockhandler, not a second P2P-side publisher, so the actor stays the sole event publisher and the write flow stays one-directional (a core requirement of the series' design).NewBlock. Import may pend the block while its parent chain is fetched, so this precedes import.block(not recency-gated), so subscribers can watch sync progress. Low-rate (one per block), so the payload is built unconditionally behindemit's no-subscriber guard.Testing
cargo fmt,cargo clippy -p ethlambda-blockchain -p ethlambda-rpc(clean), blockchain lib + rpc events tests pass.Independent PR, based off
main. One of three sibling PRs continuing the series — alongside #533 (chain_reorg/safe_target) and #534 (attestation/aggregate), each independently based offmain. They touch overlapping regions ofevents.rs/docs/rpc.md, so whichever two merge later will each need a small conflict rebase. Opened as draft.